# Self-supervised Pre-training

LWM is the first foundational model in the field of wireless communications, developed as a universal feature extractor capable of extracting fine-grained representations from wireless channel data.

MERT is an acoustic music understanding model based on self-supervised learning, using pseudo-labels provided by a teacher model for pre-training.

WavLM is a large-scale self-supervised pre-trained speech model developed by Microsoft, pre-trained on 16kHz sampled speech audio, suitable for full-stack speech processing tasks.

TAPAS is a Transformer-based table question answering model pre-trained in a self-supervised manner on English Wikipedia table data, supporting table QA and entailment tasks.